MDL Based Fitness Functions for Learning Parsimonious Programs1

Genetic programming starts with an initial population of computer programs composed of elementary functions and termir_als ~9, 10, 8]. Genetic operators, such as crossover and selection, are used to adapt the shape and size of the programs and evolve increasingly fit populations. This process can be viewed as a search for a highly fit computer program, Abe•t, in the whole program space A= {A1, A2, ... }. The quality of each computer program is measured by running it over a training set lJ of input-output cases of the unknown process !: D = {(xc, Yc)}~1 , where Xc EX, Yc E Y andyc = }(xc). The domain X and the range Yare defined by the application. The goodness of the program, A, is usually measured in terms of the error: 2:~1 (YcfA(xc)) 2 , where fA is the function realized by A. Though this training accuracy can be used as a single measure for fitness, many empirical studies have shown that, as programs grow, it also become less and less likely for them to be general [7, 15]. In addition, large structures require more computer resources in space and time for the evolution. Several methods have been proposed to take into consideration of structural complexity in fitness evaluation, but relatively few attempts have been made in the genetic programming community to employ the complexity penalty in a more principled way. In this paper we use a Bayesian model-comparison method to develop a framework in which a class of fitness measures is introduced for dealing with problems of parsimony based on the